The apparatus, systems, and methods herein utilize machine vision techniques to track locations and objects being viewed by an observer. Gaze tracking algorithms can be considered as requiring two continuous data streams in order to produce accurate tracking results: 1) eye tracking methods to detect the edges of pupils or other identifiable reference points within the eye to compute pivot angles and viewing directions of the eye, and 2) head tracking methods to locate the position and orientation of the head within our three-dimensional world.
Generally, head tracking can involve identifying the location of a rigid object affixed to the head (as opposed to the head itself). In this case, headwear or eyewear that is affixed to the head has known geometries and displacements relative to the head or reference points on the head that can be computed. More specifically, for accurate gaze tracking, a head tracking apparatus should have a known displacement from the pivot point of one or both eyeballs of the observer. Furthermore, for most applications, gaze tracking locations are determined relative to reference locations or objects within the environment of a device wearer, such as the corners of a display monitor, a mobile computing device, a switch, a light source, a window, and the like.
Applications that involve machine vision are becoming increasingly common-place. In part, this has arisen as a result of technological advances in the electronics and software development industries, and decreases in the cost of cameras, information processing units, and other electronics components. Gaze tracking, in particular, is increasingly being used in a number of diagnostic, human performance, and control applications. A small number of examples include monitoring the degree of fatigue of an individual, assessing driver or pilot awareness, assessing the effects of drugs or alcohol, diagnosing post-traumatic stress disorder, tracking human performance with age, determining the effectiveness of training or exercise, assessing the effectiveness of advertising and web-page design by measuring ocular dwell times, magnifying or changing the brightness of specific objects or images (including words) under observation, controlling various aspects of games, acquiring foundational clinical data to assess neurological or cognitive disorders, diagnosing and monitoring degenerative eye conditions, and allowing individuals with limited or no mobility below the neck to communicate by controlling a computer cursor using one or more eyes and eyelids. Sectors and industries that utilize gaze tracking include military, medicine, security, human performance, sports medicine, rehabilitation engineering, police, research laboratories, and toys.
In almost all cases, an increase in the accuracy of gaze tracking leads to an increase in the performance and convenience of most applications. For example, with increased accuracy, ocular dwell times to quantify fixation times on smaller objects or components of objects can be more accurately measured. Gaze tracking can be more effectively employed with portable devices that utilize smaller screens including mobile phones and hand-held displays. When gaze tracking is used to control a cursor involving selection from a number of virtual objects or icons within a screen, an increased number of selectable objects can be displayed simultaneously because of the ability to use smaller virtual objects or icons. An increased number of objects within each level of a selection process has a dramatic effect on the efficiency (i.e., reduced number of selection levels and/or reduced time) that a virtual object and associated action can be chosen. Similarly, enlarging or increasing the brightness levels of objects and words under observation can significantly increase recognition and reading rates of individuals who are visually impaired.
Many gaze tracking systems use cameras and eye illuminators that are located at a considerable distance (e.g., greater than ten centimeters (10 cm)) from an eye. As the distance away from the eyes is increased, an eye tracking apparatus generally becomes less obtrusive; however, it becomes increasingly difficult to accurately measure the location of an eye because of the need for higher spatial resolution by cameras and because wide-ranging head movement can cause the complete loss of the ability to track an eye. Many gaze tracking systems also use bright (visible or invisible) “point” sources of light located some distance from the head to produce glints or bright spots on the surface of the eye. These glints can be used to generate reference vectors from the location of the glint on the surface of the eye to known locations in the environment (i.e., the light sources). Here again, wide-ranging movements of the head can cause loss of the ability to track glints and/or the ability to associate a glint with a particular light source.
With the advent of modern-day microelectronics and micro-optics, it is possible to unobtrusively mount the components for gaze tracking on eyewear (e.g., eyeglasses frames) or headwear (e.g., helmet, mask, goggles, virtual reality display) including those devices disclosed in U.S. Pat. No. 6,163,281, 6,542,081, or 7,488,294, 7,515,054, the entire disclosures of which are expressly incorporated by reference herein. Using high-precision micro-optics within the eyewear or headwear, it is possible to more clearly resolve structures and reflections within the eye and nearby regions, as well as the scene viewed by the device wearer. The use of low-power, miniature cameras and electronics permits a head-mounted system to optionally be non-tethered through the use of a battery power source. Furthermore, recent advances in wireless telecommunications allow gaze tracking results to be transmitted in real-time to other computing, data storage, or control devices. As a result of these technological advances in a number of fields, an eyewear- or headwear-based gaze tracking system can be unobtrusive, light-weight, portable and convenient to use.